On Optimizing Human-Machine Task Assignments
نویسندگان
چکیده
When crowdsourcing systems are used in combination with machine inference systems in the real world, they benefit the most when the machine system is deeply integrated with the crowd workers. However, if researchers wish to integrate the crowd with “off-the-shelf” machine classifiers, this deep integration is not always possible. This work explores two strategies to increase accuracy and decrease cost under this setting. First, we show that reordering tasks presented to the human can create a significant accuracy improvement. Further, we show that greedily choosing parameters to maximize machine accuracy is sub-optimal, and joint optimization of the combined system improves performance.
منابع مشابه
Assignment of Task Modules in Hypercube
The problem of assigning task modules within a hypercube multicomputer with possible link failures is investigated. A concept of indirect optimization is introduced and a function, called communication traac, is proposed as the objective of optimization. The assignments obtained from optimizing this function are shown to signiicantly improve the actual communication performance measure, called ...
متن کاملGeneralized Task Markets for Human and Machine Computation
We discuss challenges and opportunities for developing generalized task markets where human and machine intelligence are enlisted to solve problems, based on a consideration of the competencies, availabilities, and pricing of different problemsolving resources. The approach couples human computation with machine learning and planning, and is aimed at optimizing the flow of subtasks to people an...
متن کاملSuggestion Pattern for Task-Based Curriculum design in Nursing Master’s Degree according to Aker
Introduction: The assignment-based approach is an educational strategy that emphasizes learning by doing tasks. In this approach, the learner learns how to act in different situations, and this approach teaches skills that are similar to the tasks that the learner encounters in the real world and in practice. Therefore, the purpose of this study was to design an optimal assignment-based curricu...
متن کاملHuman errors identification in operation of meat grinder using TAFEI technique
Background: Human error is the most important cause of occupational and non-occupational accidents. Because, it seems necessary to identify, predict and analyze human errors, and also offer appropriate control strategies to reduce errors which cause adverse consequences, the present study was carried out with the aim of identifying human errors while operating meat grinder and offer sugg...
متن کاملFindings of the 2011 Workshop on Statistical Machine Translation
This paper presents the results of the WMT11 shared tasks, which included a translation task, a system combination task, and a task for machine translation evaluation metrics. We conducted a large-scale manual evaluation of 148 machine translation systems and 41 system combination entries. We used the ranking of these systems to measure how strongly automatic metrics correlate with human judgme...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- CoRR
دوره abs/1509.07543 شماره
صفحات -
تاریخ انتشار 2015